UnicodeDecodeError: 'ascii' codec can't decode byte 0xec in position 2: ordinal not in range(128)

[anonymous]

Trac[main] DEBUG: Dispatching <Request "GET u'/blog'">
Trac[web_ui] DEBUG: Blog debug: command=, pagename=, path_items=[]
18시 16분 38초 Trac[model] DEBUG: get_blog_posts() SQL: SELECT bp1.name, bp1.version, bp1.publish_time, bp1.author, bp1.title, bp1.body, bp1.categories FROM fullblog_posts bp1 ,(SELECT name, max(version) AS ver FROM fullblog_posts GROUP BY name) bp2 WHERE bp1.version = bp2.ver AND bp1.name = bp2.name  ORDER BY bp1.publish_time DESC
Traceback (most recent call last):
  File "logging/__init__.py", line 744, in emit
    msg = self.format(record)
  File "logging/__init__.py", line 630, in format
    return fmt.format(record)
  File "logging/__init__.py", line 421, in format
    s = self._fmt % record.__dict__
UnicodeDecodeError: 'ascii' codec can't decode byte 0xec in position 2: ordinal not in range(128)
18시 16분 38초 Trac[model] DEBUG: get_blog_posts() SQL: SELECT bp1.name, bp1.version, bp1.publish_time, bp1.author, bp1.title, bp1.body, bp1.categories FROM fullblog_posts bp1 ,(SELECT name, max(version) AS ver FROM fullblog_posts GROUP BY name) bp2 WHERE bp1.version = bp2.ver AND bp1.name = bp2.name  ORDER BY bp1.publish_time DESC
Traceback (most recent call last):
  File "logging/__init__.py", line 744, in emit
    msg = self.format(record)
  File "logging/__init__.py", line 630, in format
    return fmt.format(record)
  File "logging/__init__.py", line 421, in format
    s = self._fmt % record.__dict__
UnicodeDecodeError: 'ascii' codec can't decode byte 0xec in position 2: ordinal not in range(128)
Trac[chrome] DEBUG: Prepare chrome data for request
Trac[session] DEBUG: Retrieving session for ID u'blog'
Trac[model] DEBUG: get_blog_comments() SQL: SELECT name, number, comment, author, time FROM fullblog_comments WHERE name=%s
Trac[__init__] ERROR: 'ascii' codec can't decode byte 0xec in position 2: ordinal not in range(128)
Traceback (most recent call last):
  File "/workspace/trac/trac-current/trac/web/main.py", line 381, in dispatch_request
    dispatcher.dispatch(req)
  File "/workspace/trac/trac-current/trac/web/main.py", line 213, in dispatch
    data, content_type)
  File "/workspace/trac/trac-current/trac/web/chrome.py", line 601, in render_template
    return stream.render(method, doctype=doctype)
  File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/core.py", line 154, in render
    return encode(generator, method=method, encoding=encoding)
  File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 45, in encode
    output = u''.join(list(iterator))
  File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 274, in __call__
    for kind, data, pos in stream:
  File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 519, in __call__
    for kind, data, pos in stream:
  File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 691, in __call__
    text = escape(pop_text(), quotes=False)
  File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/core.py", line 405, in escape
    text = unicode(text).replace('&', '&amp;') \
UnicodeDecodeError: 'ascii' codec can't decode byte 0xec in position 2: ordinal not in range(128)
Trac[chrome] DEBUG: Prepare chrome data for request
127.0.0.1 - - [29/Oct/2007 18:16:41] "GET /aide/blog HTTP/1.1" 500 

[anonymous]

same problem for Chinese version

[osimons]

Patch to disable all logging - it seems to be the reason for unicode issues.

[osimons]

From the errors, it actually seems to be a problem with one (or more) of my debug statements that don't convert properly. To check this, I would very much like someone to apply the remove-logging-r2709.diff​ attachment - it basically comments out all debug statements.

Does the plugin work correctly after patching?

[anonymous]

applied patch but:

14시 18분 00초 Trac[main] DEBUG: Dispatching <Request "GET u'/blog'">
14시 18분 00초 Trac[chrome] DEBUG: Prepare chrome data for request
14시 18분 01초 Trac[session] DEBUG: Retrieving session for ID u'benn'
14시 18분 01초 Trac[init] ERROR: 'ascii' codec can't decode byte 0xec in position 2: ordinal not in range(128)
Traceback (most recent call last):

File "/workspace/trac/trac-current/trac/web/main.py", line 381, in dispatch_request

dispatcher.dispatch(req)

File "/workspace/trac/trac-current/trac/web/main.py", line 213, in dispatch

data, content_type)

File "/workspace/trac/trac-current/trac/web/chrome.py", line 608, in render_template

return stream.render(method, doctype=doctype)

File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/core.py", line 154, in render

return encode(generator, method=method, encoding=encoding)

File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 45, in encode

output = u.join(list(iterator))

File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 274, in call

for kind, data, pos in stream:

File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 519, in call

for kind, data, pos in stream:

File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 691, in call

text = escape(pop_text(), quotes=False)

File "/usr/local/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/core.py", line 405, in escape

text = unicode(text).replace('&', '&amp;') \

UnicodeDecodeError: 'ascii' codec can't decode byte 0xec in position 2: ordinal not in range(128)
14시 18분 01초 Trac[chrome] DEBUG: Prepare chrome data for request
127.0.0.1 - - [02/Nov/2007 14:18:02] "GET /aide/blog HTTP/1.1" 500 -
14시 18분 03초 Trac[main] DEBUG: Dispatching <Request "GET u'/chrome/common/css/trac.css'">
127.0.0.1 - - [02/Nov/2007 14:18:03] "GET /aide/chrome/common/css/trac.css HTTP/1.1" 304 -
14시 18분 03초 Trac[main] DEBUG: Dispatching <Request "GET u'/chrome/common/js/jquery.js'">
127.0.0.1 - - [02/Nov/2007 14:18:03] "GET /aide/chrome/common/js/jquery.js HTTP/1.1" 304 -
14시 18분 04초 Trac[main] DEBUG: Dispatching <Request "GET u'/chrome/common/js/trac.js'">
127.0.0.1 - - [02/Nov/2007 14:18:04] "GET /aide/chrome/common/js/trac.js HTTP/1.1" 304 -
14시 18분 04초 Trac[main] DEBUG: Dispatching <Request "GET u'/chrome/common/js/search.js'">
127.0.0.1 - - [02/Nov/2007 14:18:04] "GET /aide/chrome/common/js/search.js HTTP/1.1" 304 -
14시 18분 05초 Trac[main] DEBUG: Dispatching <Request "GET u'/chrome/common/trac_logo_mini.png'">
127.0.0.1 - - [02/Nov/2007 14:18:05] "GET /aide/chrome/common/trac_logo_mini.png HTTP/1.1" 304 -
14시 18분 05초 Trac[main] DEBUG: Dispatching <Request "GET u'/chrome/common/trac_banner.png'">
127.0.0.1 - - [02/Nov/2007 14:18:05] "GET /aide/chrome/common/trac_banner.png HTTP/1.1" 304 -
14시 18분 05초 Trac[main] DEBUG: Dispatching <Request "GET u'/chrome/common/dots.gif'">
127.0.0.1 - - [02/Nov/2007 14:18:05] "GET /aide/chrome/common/dots.gif HTTP/1.1" 304 -
14시 18분 05초 Trac[main] DEBUG: Dispatching <Request "GET u'/chrome/common/topbar_gradient.png'">
127.0.0.1 - - [02/Nov/2007 14:18:05] "GET /aide/chrome/common/topbar_gradient.png HTTP/1.1" 304 -
14시 18분 05초 Trac[main] DEBUG: Dispatching <Request "GET u'/chrome/common/topbar_gradient2.png'">
127.0.0.1 - - [02/Nov/2007 14:18:05] "GET /aide/chrome/common/topbar_gradient2.png HTTP/1.1" 304 -

[osimons]

Been trying to figure this out, but I can't find any issues with rendering international characters on my setup. 4 ideas:

1. This is a clean, recent Trac 0.11dev you are testing on, right? If you have made modifications to site.html try renaming the file to disable it temporarily and restart the server to see if something in this file is affecting output.
2. If you have set content for the info-area (through Admin), try clearing that out to see if it is the rendering of that text that is the problem.
3. Does it work at all if you create a new project, enable the plugin and browse it without any posts in the database at all?
4. Finally it could be related directly to your content in the posts. Could you perhaps make a minimal example set of blog posts and comments that make the error appear, and send me a dump of the database? If the database is just example data, you can attach it to this ticket, alternatively send it to me by email (see osimons for address). If you use sqlite, simply dump by doing:

   sqlite3 /path/to/project/db/trac.db .dump > blog.dump
   

[anonymous]

I am using a Chinese version of MS-Windows. Same problem for me. The error occurred when I submit the blog (both error through preview and submit). However, the error begins to occur when I click on the blog menu and could not even enter into the blog page.

[osimons]

Ok. Then I suppose it must be something with the content format/encoding when rendering posts. I can't provoke it with my European-based international characters, and on my OSX development machine I've tried entering posts with all fields from a variety of Asian languages. They all work.

If someone could help me out with an example sqlite database where the error is provoked, it would be most helpful:

1. create a new trac project - trac-admin /path/to/new/project/ initenv
2. install/enable the blog plugin
3. open project, and submit a sample post that provokes the error
4. close down server, and send me the trac.db file for the project as-is.

[osimons]

Found it - thanks anonymous :-)

Fixed in [2760].